Tackling Neonatal Sepsis—Can It Be Predicted?

(1) Background: Early signs of sepsis in a neonate are often subtle and non-specific, the clinical course rapid and fulminant. The aim of our research was to analyse diagnostic markers for neonatal sepsis and build an application which could calculate its probability. (2) Methods: A retrospective clinical study was conducted on 497 neonates treated at the Clinical Department of Neonatology of the University Children’s Hospital in Ljubljana from 2007 to 2021. The neonates with a diagnosis of sepsis were separated based on their blood cultures, clinical and laboratory markers. The influence of perinatal factors was also observed. We trained several machine-learning models for prognosticating neonatal sepsis and used the best-performing model in our application. (3) Results: Thirteen features showed highest diagnostic importance: serum concentrations of C-reactive protein and procalcitonin, age of onset, immature neutrophil and lymphocyte percentages, leukocyte and thrombocyte counts, birth weight, gestational age, 5-min Apgar score, gender, toxic changes in neutrophils, and childbirth delivery. The created online application predicts the probability of sepsis by combining the data values of these features. (4) Conclusions: Our application combines thirteen most significant features for neonatal sepsis development and predicts the probability of sepsis in a neonate.


Introduction
Sepsis is a potentially life-threatening syndrome of organ dysfunction induced by a dysregulated response to an infection [1]. The risk of developing sepsis is especially high in neonates due to their vulnerable and underdeveloped immune system [2]. We differentiate between neonatal early-onset sepsis (EOS), which develops in the first 72 h of life, and lateonset sepsis (LOS), which develops after that. In general, EOS is most commonly caused by Group B Streptococcus (GBS) and Escherichia coli (E. coli) [3,4], whereas the primary agents of LOS appear to be Coagulase-negative Staphylococci (CoNS) and Staphylococcus aureus (S. aureus), followed by E. coli and GBS [3,5]. The onset is highly dependent on perinatal factors such as gender, gestational age, birth weight, Apgar score, type of childbirth delivery, and maternal factors [2,6]. Early signs of sepsis are often subtle and non-specific [6], ranging from temperature instability (hypothermia more often than fever) to respiratory and cardiac clinical signs, poor feeding, and lethargy [5]. Yet, the clinical course of neonatal sepsis can be contrastingly rapid and fulminant [6]. Clinical presentation is due to its variability and heterogeneity always interpreted with regard to the non-specific markers of infection, most commonly the C-Reactive Protein (CRP) and Procalcitonin (PCT), whereas blood cultures remain the gold standard for diagnosing sepsis, even with the delay in results [2]. Over the years, researchers have focused on finding a single diagnostic test or a collection of tests that would help neonatologists reach precise and fast recognition of a neonate with sepsis. Furthermore, it would help with early diagnostics, followed by specific and effective treatment [7]. The focus was predominantly on serum concentrations of CRP and PCT, the complete blood cell count (CBC), and the ratio between immature and total neutrophil count (I:T) [8]. For the early detection of neonatal sepsis, several researchers focused on combining various markers of the disease and building prediction models out of them. In 2022, Sahu et al. reviewed multiple studies on forecasting algorithms for neonatal sepsis and recognised predictors among maternal, neonatal, and laboratory parameters. In addition, a different array of promising factors was put forward for the early detection of EOS, LOS, and neonatal sepsis in general. The reviewed prediction models were mostly built by logistic regression (in 7 out of 10 studies) [9]. However, other statistic methodologies could also prove suitable for forecasting algorithms. In 2021, Neamtu et al. recognised the decision-tree approach as being useful for predicting the outcomes of neonatal brain injury. They pointed out that the decision-tree approach has also been put to test for forecasting other aetiologies, including neonatal infections [10]. Regardless of the applied algorithm, all researchers concluded that through further developments in this field, prediction modelling could have large implications for early recognition of adverse outcomes of neonates in the future [9,10]. To this day, tackling neonatal sepsis remains a topical issue for paediatricians all over the world. The incidence of neonatal sepsis on a global level is estimated at 2824 neonates per 100,000 live births, or 2.8%, largely determined by the income level of the country [11]. While its high morbidity and mortality rates are still the primary concern of the less developed countries, the developed countries swing between the antibiotic over-prescription and under-prescription on account of acting too late. With respect to that, we conducted research to obtain insight into the burden of neonatal sepsis in Slovenia and to analyse diagnostic markers used in our population. Our aim was to propose an approach that could more easily and precisely differentiate neonates with sepsis from those without sepsis. We attempted to build an application that could calculate the probability of sepsis and could hence aid clinicians in a timelier diagnosis and well-reasoned care of neonates.

Materials and Methods
A retrospective clinical study was conducted on neonates who were treated at the Clinical Department of Neonatology of the University Children's Hospital in Ljubljana from 2007 to 2021. In total, we collected data for 998 infants, but half (501) of them had missing data and were excluded. Out of the remaining 497 infants, 403 neonates met the inclusion criteria of being diagnosed with neonatal sepsis based on the International Classification of Diseases (P36.0-P36.9), and 94 represented the control group. These were the neonates treated due to other, non-infectious diseases. Patient data were collected from a paediatric computer information system called ISPEK. Data collected consisted of perinatal factors, clinical signs, and laboratory markers of infection. Observed perinatal factors were gestational age, birth weight, head circumference, gender, type of childbirth delivery, 5-min Apgar score, maternal risk factors (e.g., prolonged rupture of membranes, intrapartum antimicrobial prophylaxis, prenatal exposure to steroids), presence of a meconium-stained amniotic fluid, and the age of onset. Out of clinical signs, we observed changes in body temperature (>38.5 • C/<36.0 • C), the presence of tachycardia (>160 beats/minute), tachypnoea (>60 breaths/minute), inappetence, prolonged capillary refill (>3 s), irritability, lethargy, and jaundice. Among laboratory markers of infection, serum CRP and PCT concentrations and CBC were observed. The neonates with a diagnosis of sepsis were separated based on their blood culture results, clinical signs, and laboratory markers of infection and were later divided into three groups: proven, probable, and suspected sepsis group (Table 1). For the purpose of more accurate model training, the neonates in the proven and probable groups were defined as septic, and the neonates in the suspected sepsis and control group were defined as non-septic. Using machine learning, we developed several models in Python for predicting neonatal sepsis. We used the best-performing model as the core of our application. Out of all the observed features, we only used seventeen with the least missing values. Since missing values in our dataset had no hidden dependency on any other variable, the type of missing data was Missing Completely at Random (MCAR). To approximate them, we applied the Nearest Neighbour imputation, which is intuitive and yields good results when there are not many features and observations [12,13]. The dataset was split into training and testing sets; the training set contained 397 neonates (80%), and the testing set contained 100 (20%). The distribution of septic and non-septic neonates was equal in both sets. The training set was used to train the following models: logistic regression, decision tree classifier, Support Vector Machine, K-Nearest Neighbours, and Random Forest. We evaluated the models' efficacy with standard classification metrics: classification accuracy, sensitivity, specificity, positive and negative predictive value, Area under Curve, and F1 score. The most promising model was Random Forest (Classification and Regression Trees or CART); the others were discarded. The Random Forest algorithm comprises multiple separate decision trees. A figure of one of the decision trees used in Random Forest is in Appendix A ( Figure A1). We improved the accuracy of the Random Forest model by training it on a modified dataset, which contained only the thirteen most important features. These features were serum concentration of CRP, the age of onset, immature neutrophil percentage, serum concentration of PCT, lymphocyte percentage, leukocyte count, thrombocyte count, birth weight, gestational age, 5-min Apgar score, gender, presence of toxic changes in neutrophils, and the type of childbirth delivery. Feature selection was based on feature importance metrics. In our case, we applied feature selection methods that are part of the Random Forest algorithm. Furthermore, for fine-tuning the hyperparameters, grid search with k-fold Cross-Validation was used. We used five folds to get the best parameters. After we got the best parameters, we trained the model again with ten-fold Cross-Validation.

Results
In total, 497 neonates were included in the analysis. The percentage of neonates in the sepsis group (proven and probable sepsis) was proportionate to the number of neonates in the non-sepsis group (suspected sepsis and control group), which is 49% and 51%, respectively ( Figure 1). Using machine learning, we developed several models in Python for predicting neonatal sepsis. We used the best-performing model as the core of our application. Out of all the observed features, we only used seventeen with the least missing values. Since missing values in our dataset had no hidden dependency on any other variable, the type of missing data was Missing Completely at Random (MCAR). To approximate them, we applied the Nearest Neighbour imputation, which is intuitive and yields good results when there are not many features and observations [12,13]. The dataset was split into training and testing sets; the training set contained 397 neonates (80%), and the testing set contained 100 (20%). The distribution of septic and non-septic neonates was equal in both sets. The training set was used to train the following models: logistic regression, decision tree classifier, Support Vector Machine, K-Nearest Neighbours, and Random Forest. We evaluated the models' efficacy with standard classification metrics: classification accuracy, sensitivity, specificity, positive and negative predictive value, Area under Curve, and F1 score. The most promising model was Random Forest (Classification and Regression Trees or CART); the others were discarded. The Random Forest algorithm comprises multiple separate decision trees. A figure of one of the decision trees used in Random Forest is in Appendix A ( Figure A1). We improved the accuracy of the Random Forest model by training it on a modified dataset, which contained only the thirteen most important features. These features were serum concentration of CRP, the age of onset, immature neutrophil percentage, serum concentration of PCT, lymphocyte percentage, leukocyte count, thrombocyte count, birth weight, gestational age, 5-min Apgar score, gender, presence of toxic changes in neutrophils, and the type of childbirth delivery. Feature selection was based on feature importance metrics. In our case, we applied feature selection methods that are part of the Random Forest algorithm. Furthermore, for fine-tuning the hyperparameters, grid search with k-fold Cross-Validation was used. We used five folds to get the best parameters. After we got the best parameters, we trained the model again with ten-fold Cross-Validation.

Results
In total, 497 neonates were included in the analysis. The percentage of neonates in the sepsis group (proven and probable sepsis) was proportionate to the number of neonates in the non-sepsis group (suspected sepsis and control group), which is 49% and 51%, respectively ( Figure 1).

Pathogens
The pathogens, isolated from the investigated blood cultures, were Gram-positive (G+) and Gram-negative (G-) bacteria. Neonatal sepsis was more often caused by G+ bacteria (in 52 neonates, or 57%) and was predominantly late onset (in 80 neonates, or 88%). The most common pathogens were E. coli and GBS, which were isolated in 27% and 25% of all blood cultures, respectively, followed by CoNS in 15% and S. aureus in 13%. The distribution of causative agents is shown in Figure 2. We isolated S. epidermidis and S. haemolyticus among CoNS, E. faecalis and E. faecium from Enterococcus spp., K. oxytoca and K. pneumoniae from Klebsiella spp., E. cloacae from Enterobacter spp., and placed Serratia marscensens and Acinetobacter junii among other G-bacteria.

Pathogens
The pathogens, isolated from the investigated blood cultures, were Gram-positive (G+) and Gram-negative (G-) bacteria. Neonatal sepsis was more often caused by G+ bacteria (in 52 neonates, or 57%) and was predominantly late onset (in 80 neonates, or 88%). The most common pathogens were E. coli and GBS, which were isolated in 27% and 25% of all blood cultures, respectively, followed by CoNS in 15% and S. aureus in 13%. The distribution of causative agents is shown in Figure 2. We isolated S. epidermidis and S. haemolyticus among CoNS, E. faecalis and E. faecium from Enterococcus spp., K. oxytoca and K. pneumoniae from Klebsiella spp., E. cloacae from Enterobacter spp., and placed Serratia marscensens and Acinetobacter junii among other G-bacteria.

Perinatal Factors
Neonates included in our study did not differ much in the general perinatal characteristics, such as gestational age, birth weight (mostly term), and head circumference (Table 2). They were mostly male and were more often delivered by vaginal birth. Proven sepsis was most commonly of late onset. Though the median 5-Minute Apgar score was nine in all groups, it can be seen from the interquartile range that the neonates in the sepsis group were more likely to require immediate medical attention after birth than those in the non-sepsis one. Based on our findings, a meconium-stained amniotic fluid or maternal factors did not act as important risk factors for developing neonatal sepsis.

Perinatal Factors
Neonates included in our study did not differ much in the general perinatal characteristics, such as gestational age, birth weight (mostly term), and head circumference ( Table 2). They were mostly male and were more often delivered by vaginal birth. Proven sepsis was most commonly of late onset. Though the median 5-Minute Apgar score was nine in all groups, it can be seen from the interquartile range that the neonates in the sepsis group were more likely to require immediate medical attention after birth than those in the non-sepsis one. Based on our findings, a meconium-stained amniotic fluid or maternal factors did not act as important risk factors for developing neonatal sepsis.

Clinical Presentation of Neonatal Sepsis
Neonates with sepsis did not exhibit a specific clinical presentation, nor were the clinical signs of infection often present. Even though almost all the investigated clinical signs were more commonly observed in septic neonates, they were still not very frequent, as only one reached above 50% (Table 3). By definition, the neonates in the control group did not have any clinical signs of infection.

Laboratory Markers of Infection
The median leukocyte count did not differ much in septic and non-septic neonates, but it can be seen from the interquartile range that septic neonates had lower leukocyte counts. Neonates with sepsis had a higher percentage of immature and mature neutrophils, as well as higher I:M and I:T ratios, in addition to lower lymphocyte percentage. Thrombocyte count was highest in the control group. Toxic changes in neutrophils were frequently present in the sepsis group. The levels of serum CRP and PCT concentrations were higher in septic neonates (Table 4).   Table 5 shows standard classification metrics for all trained models. Based on classification metrics, the best-performing model was Random Forest (Table 6). This model was later embedded into an online application. Table 6. Classification metrics of the best performing Random Forest model after fine-tuning.  Figure 3 shows thirteen out of seventeen features that had the highest importance values based on feature selection methods. They were selected for model training. The best-performing model was used for our application, which allows the user to enter the patient's data of those features and calculates the probability of sepsis in the neonate. The application can be accessed at https://sepsa.blang.eu/ (accessed on 26 December 2022).

CA (%)
Se (%) Sp (%) PPV (%) NPV (%) F1 (%) AUC (  m Forest  83  81  87  86  79  84  84  c Regression  81  76  86  86  74  80  81  n Tree Classifier  82  80  84  84  80  82  82  rt Vector Machine  79  75  83  83  75  78  79  rest Neighbours  62  35  78  78  35  Based on classification metrics, the best-performing model was Random Forest ble 6). This model was later embedded into an online application.   Figure 3 shows thirteen out of seventeen features that had the highest import values based on feature selection methods. They were selected for model training. best-performing model was used for our application, which allows the user to ente patient's data of those features and calculates the probability of sepsis in the neonate. application can be accessed at https://sepsa.blang.eu/ (accessed on 26 December 2022

Discussion
The main goal of this research was to determine whether neonatal sepsis can so how be quantified, and therefore, predicted, even in the early stages of its developm

Discussion
The main goal of this research was to determine whether neonatal sepsis can somehow be quantified, and therefore, predicted, even in the early stages of its development. Firstly, we established which perinatal factors act as risk factors for developing sepsis, which clinical signs, if any, are the most indicative of the disease, and which diagnostic markers of infection are both the most common and most significant for reaching the diagnosis. We focused on data that are easily obtained and observed and could act as a significant as well as an early marker of neonatal sepsis. Secondly, we classified features based on their importance for predicting neonatal sepsis and embedded a selected combination into an online application. The inclusion criteria were commonness and feature importance or significance in neonatal sepsis prediction. In addition, the number of selected features had to be just right to not cause overfitting of the prediction model. After entering the patient's data into all the required fields, the application calculates the probability of sepsis in that specific neonate. The result is a percentage, and a higher percentage corresponds to a higher chance of a neonate being septic.
Based on our findings, five perinatal risk factors for neonatal sepsis were included in our model. Those were male gender, vaginal delivery, lower 5-min Apgar score (<7), lower gestational age, and low birth weight (LBW) (Figure 3). The male gender appeared to be more vulnerable to developing neonatal sepsis (Table 2), which was put forward by other researchers as well [3,4]. However, there were more male neonates in the non-sepsis group as well, which could imply that in general, males are of greater susceptibility in the days following their birth, compared to females. Furthermore, the risk of EOS was greater if the mother was colonised with GBS, which is recognized as the primary pathogen responsible for EOS [4]. Thus, neonates delivered vaginally are generally at higher risk of developing sepsis due to vertical (mother-neonate) transmission of pathogens [14]. A lower Apgar score indicates that the neonate needs immediate medical attention after birth and can be, to some extent, associated with a greater risk of infection [2,[15][16][17][18]. Preterm and LBW neonates are more susceptible to infections due to their less mature immune system and the deficit of protective maternal IgG antibodies that cross the placenta of term neonates [3,6]. In our study, LOS was more common than EOS (the median onset time was 9.5 days). On one hand, the reason could lie behind efficient antenatal screening for GBS; on the other hand, the neonates were most often admitted from their homes (out-of-hospital setting), which could contribute to a higher level of LOS cases. To continue, none of the observed clinical signs turned out as reliable indicators for sepsis, which is in line with the findings of other researchers. The clinical presentation is often referred to as non-specific, as an ill-looking neonate can be potentially septic. However, it could also have meningitis, pneumonia, or suffer from prematurity or asphyxia-related complications [19]. Hence, the clinical presentation was not included in the application. Nevertheless, we recognize that in our research, the lower reliability of clinical presentation might be related to the overall lower frequency of observed signs of infection in neonates (Table 3). This could be due to the error of subjective assessment and insufficient anamnestic data of parents whose neonates developed sepsis outside the hospital. Out of all the laboratory values tested, only the counts of the leukocytes and thrombocytes, the percentages of immature neutrophils and lymphocytes, the presence of toxic changes in neutrophils, and the serum concentrations of CRP and PCT acted as significant for diagnosing neonatal sepsis (Figure 3). In a neonate, CBC must be interpreted carefully concerning the clinical signs and other laboratory markers of infection. Contrary to older children and adults, the normal range of the total white blood cell count in a neonate is quite wide (9-30 × 10 9 /L); hence, it does not act as a precise tool for diagnosing an infection [20,21]. In our study, leukocyte count showed an importance of only 0.06 (Figure 3). The leukocyte count in a neonate is generally on the high-value side on the first day of life and decreases later [22]. Based on the literature, a low leukocyte count is associated with EOS as well as LOS, and a high leukocyte count is associated only with LOS [23,24]. While interpreting leukocyte count, we should take into consideration that the neutrophil count tends to be lower at lower gestational age and that other clinical conditions can also affect it (perinatal asphyxia, maternal fever, meconium aspiration syndrome, and delivery route) [22]. Out of all haematological features included in model training, immature neutrophil percentage showed the highest importance value (0.11) (Figure 3). The medians of immature neutrophil percentages and I:T ratios were clearly higher in septic neonates ( Table 4). The I:T ratio may be, based on the literature, the most sensitive indicator of neonatal sepsis out of all haematological indices [22]. In addition, during an immune response to an infection, changes in lymphocyte count also occur [25]. Lymphocyte count showed an importance value of 0.08 with its median lower in septic neonates (Table 4). However, we can mention that nowadays, the neutrophil/lymphocyte ratio is becoming a more promising inflammation marker than lymphopenia. It has a significant positive correlation with EOS and will surely be the subject of future studies [25]. Furthermore, certain toxic changes in leukocytes, such as cytoplasmic vacuolization and toxic granulations, can occur as a result of an infection. These changes are generally considered late markers of infection and are never present in healthy neonates [26][27][28]. In our study, they were more frequently present in the septic group of neonates (Table 4). Besides changes in leukocyte count and toxic changes in leukocytes, thrombocytopenia also appeared to be related to increased odds of infection in our research. This finding is similar to other studies [23,24]. To continue, CRP is an acute-phase protein and the most commonly used marker of infection in neonates. It is synthesised in the liver in response to infection or tissue damage [26,29]. In our study, the serum concentration of CRP acted as the most important feature for diagnosing sepsis with the importance value of 0.37 ( Figure 3). Its values were much higher in the sepsis group in comparison to the non-sepsis group, where the median was 0 ( Table 4). Despite that, we must be careful at interpreting its concentration in the first days after birth because of different antenatal and perinatal factors (e.g., prolonged rupture of membranes, intrapartum antimicrobial prophylaxis, prenatal exposure to steroids), which can also affect its values [30]. In addition to that, meconium aspiration syndrome, surgical procedures, chorioamnionitis, and perinatal anoxia can also cause higher serum CRP concentration [31][32][33]. With that in mind, the neonates with other valid reasons for high values of CRP, such as severe birth asphyxia, viral infections and severe brain haemorrhage, were excluded from our study. Moreover, PCT also acts as an acute-phase protein whose serum concentration increases with the gravity of the infection. Contrary to the serum concentration of CRP, local bacterial infections, and viral and other non-bacterial infections result in a lower increase in the serum concentration of PCT [20,[34][35][36]. While interpreting laboratory results in the first 24 h after birth, we have to keep in mind a physiological increase in serum PCT concentration [30]. Based on our findings, the median serum PCT concentration was higher in the sepsis group and around 0 in the non-sepsis group, similar to serum CRP concentration ( Table 4). The importance of serum PCT concentration was, on the other hand, lower (0.1) (Figure 3). Around 20% of the values of serum PCT concentration were missing and we had to apply imputation techniques to approximate them, which could have had some impact on the results. Nevertheless, serum PCT concentration is, based on the literature, more relevant for diagnosing the early stages of neonatal sepsis than serum CRP concentration [37]. It would be interesting to compare our importance values with a similar, potentially prospective study, which would include more measurements of serum PCT, as it is being more regularly monitored nowadays.
In comparison, Kaiser Permanente developed a Neonatal Early-Onset Sepsis Calculator which is easily accessible online [38]. It has been recognized as a convenient tool that significantly impacts patient care and reduces antibiotic administration, the number of laboratory tests, and admissions to the neonatal unit without increasing mortality or readmissions of the neonates [39,40]. In this calculator, the probability of neonatal sepsis is calculated based on the values of perinatal risk factors. The probability is given as the risk of EOS per 1000 births, which can also be modified considering the neonate's clinical presentation. There are several strengths of the calculator. Firstly, the required data are usually quite easily accessible, unlike the results of the laboratory markers, which take several hours. Secondly, after calculating the risk of developing neonatal sepsis, the calculator also recommends further management of a neonate. Thirdly, Kaiser's solution of minimising the impact of a neonate's immaturity due to preterm birth was setting the bar at 34 gestation weeks, which could be taken into consideration when making further improvements to our application. On the other hand, Kaiser's calculator only predicts the risk of developing EOS, and not EOS and LOS combined, as it is in our case. The advantage of including all septic neonates, regardless of the onset time of the disease, is the broader use of such an application. However, we do recognize that higher applicability could come on the account of a lower precision of the calculated probability of sepsis. In light of our work, Kaiser's calculator also lacks the values of laboratory markers of infection such as the serum concentrations of CRP and PCT, and CBC. Their reasoning behind this omission is that the serum concentration of CRP has poor positive predictive value and that it has not been proven to be useful for diagnosing sepsis in asymptomatic neonates. In addition, observing CBC is only recommended as an additional test and the serum concentration of PCT is not mentioned at all. We could argue that, though serum concentration of CRP is in fact a late marker of infection, it still adds value to the calculation of neonatal sepsis prediction, especially when considered alongside other laboratory markers of infection. In our study, the serum concentration of CRP proved to be the most important feature associated with sepsis, though its importance value was still only 0.37. However, if we add the importance values of serum PCT concentration and the percentage of immature neutrophils, which appeared as the most important features (Figure 3) besides serum CRP concentration, the end value of feature importance is 0.58. Furthermore, the value increases if we add even more features, which are significant for diagnosing neonatal sepsis; however, adding too much could be a double-edged sword leading to overfitting of the model. Finding the balance and just the right number of features were our goals when creating the application. Nevertheless, we do acknowledge the effect of the predetermined cut-off values of the laboratory markers and that their adjustment or finer tuning could potentially lead to more accurate results. Lastly, in the pursuit of predicting neonatal sepsis as precisely as possible, the two calculators could perhaps be united; the number of required features would thus be expanded or even reduced.
The main limitation of our research is that the study was retrospective. That allowed us to get quite a big sample of almost a thousand neonates; however, only approximately half of them were included in the final analysis due to missing values. Moreover, the included neonates had missing data as well, which was the reason for using the Nearest Neighbour imputation. Even though imputation is an established approach in machine learning, we acknowledge that it can increase variance and cause overfitting. It is also worth mentioning that obtaining a sufficient amount of blood for analysis purposes can be challenging due to the neonates' lower total blood volume. In addition, as every neonate needs to be treated individually and the selection of medical tests chosen rationally, the selection of laboratory markers of infection amongst the patients differed. While the serum concentration of CRP was examined almost routinely, it was the serum concentration of PCT that was most frequently lacking. In addition, since the study was retrospective, we could not determine the significance of newly proposed biomarkers for sepsis, such as presepsin [2]. Another limitation of our research is not collecting data of blood urea nitrogen (BUN) and creatinine. Based on recent developments, BUN is strongly associated with the presence and severity of neonatal sepsis [41]. Monitoring creatinine could be useful because septic neonates are at higher risk of developing renal dysfunction and acute kidney injury [42]. However, it is our belief that all this could be field for further research. Should the findings be relevant for prognosticating neonatal sepsis, they could be integrated into our application. Further studies could also collect data on term and preterm neonates separately and determine how maturity and immaturity shape the neonates' response to an infection.

Conclusions
To conclude, the ideal diagnostic marker of neonatal sepsis is still not known. In practice, a diagnosis of sepsis in neonates is reached by combining perinatal risk factors, clinical signs and laboratory markers of infection, and the results of blood cultures. Gathering all the necessary markers can be time-consuming, but an early management of a potentially septic neonate is contrastingly crucial. In light of aiding clinicians in reaching a timely and more precise diagnosis of neonatal sepsis, we created an application that predicts its probability. The application can hopefully prove useful to neonatologists when handling potentially septic neonates and impact patient care altogether. We believe our work shows a promising step towards prediction modelling in the field of neonatal sepsis; however, extensive research and further developments are still much needed.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the MEDICAL ETHICS COMMITTEE OF THE REPUBLIC OF SLOVENIA (protocol code 0120-518/2020/6 and date of approval 3 February 2021).

Informed Consent Statement:
We conducted a retrospective study, which was first approved by the Medical Ethics Committee of the Republic of Slovenia.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
The Random Forest algorithm was used to develop a model and web application to predict if neonates have sepsis or not. It is a classification technique based on the decision tree method. The Random Forest algorithm comprises multiple separate decision trees, which are merged to increase the accuracy and stability of data. There were fifty decision trees in our case. In the process of analysis, each tree gives a classification (a "vote"), and the algorithm chooses the classification that receives the most "votes". There is no correlation between the trees because the features, included in the trees, are generated randomly. Every tree has its own subset of features and not every tree consists of all possible features. With the bootstrapping method, data sampling and replacement are performed. The end calculation consists of a random sample of all data. The accuracy of our best-performed model in the Random Forest was 86%, sensitivity 82%, specificity 89%, positive predictive value 89%, negative predictive value 82%, and F1 score 85%. Figure A1 shows the decision tree number 0. It is an example of a decision tree used in our Random Forest model. The sample size/training set was 397 neonates. The root node in a decision tree is our starting point. With every next split, we aim to maximize the number of units in each division according to the most distinguishable values between the classes.